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ABSTRACT 


AN  EXPERIMENT  ON  EXECUTIVE  DECISION  MAKING 

by 

Eduard  J.  Fidler  and  Gerald  L.  Thompson 
Carnegie-Melion  University 

This  paper  analyzes  capital  budgeting  decisions  of  business  executives 
in  an  experimental  setting.  Tvo  differer.i.  data  analysis  techniques,  a 
linear  programming  and  a maximum  likelihood  method,  are  used  to  analyze 
preference  juugu.onts  for  hypothetical  investment  projects.  For  each  of 
the  twenty  investment  projects  the  following  information  was  provided:  a 
probability  distribution  of  returns  on  investment,  the  magnitude  of  the 
investment,  the  number  of  additional  men  to  be  hired  for  implementing  the 
investment,  and  the  expected  payback  period  for  the  investment.  Five  con- 
clusions are  demonstrated  based  on  two  independent  experiments  with  business 
executives: 

(1)  Mean  return  on  i 'CJtment,  and  payback  period,  are  the  most 
important  investment  characteristics; 

(2)  Upper  and  middle  management  executives  have  almost  identical 
preference  structures; 

(3)  Downside  variation  of  the  return  distribution  has  a stronger 
influence  on  investment  decisions  than  overall  variation; 

(4)  Usefulness  of  this  methodology  for  (i)  analyzing  preference 
judgments  of  individual  executives;  (ii)  training  t.  i“ooses; 
and  (iii)  as  an  aid  to  organizational  decision  making; 

(5)  Both  data  analysis  techniques  give  highly  similar  conclusions. 
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1.  INTRODUCTION 

One  of  the  most  common  executive  decisions  involves  the  acceptance 
or  rejection  of  Investment  projects.  Surprisingly,  there  have  been  Jew 
empirical  analyses  of  these  decisions.  Those  that  have  been  reported 
typically  use  survey  methods  to  determine  the  relative  use  of  different 
capital  budgeting  and  risk  evaluation  techniques.  (e.g.,  Fremgen  [8], 

Klammer  [10],  and  Petty,  Scott,  and  Bird  liS]).  Although  these  studies 
report  which  techniques  executives  use  for  evaluating  capital  Investments, 
they  do  not  examine  how  executives  combine  the  results  obtained  by  these 
techniques  into  an  overall  preference  judgment  for  investment  projects. 

Two  studies  which  have  attempted  to  address  this  latter  issue  were 
those  of  Mao  [13]  and  Conrath  [3].  In  the  former,  the  author  used  inter- 
views with  business  executives  in  eight  medium  and  large  companies  to  investi- 
tate  (1)  the  risk  analysis  employed  in  investment  decisions  and  (2)  the 
criteria  for  investment  selection.  Concerning  the  concept  of  risk  Mao  re- 
ported that  "...when  the  investment  decision  involves  only  a small  portion  of 
the  resources  of  the  company,  risk  is  primarily  considered  to  be  the  prospect 
of  not  meeting  some  target  rate  of  return.  However,  when  the  investment 
concerns  a large  portion  of  the  company's  resources,  risk  also  involves  the 
danger  of  insolvency."  [p.354].  Risk  is  incorporated  into  the  selection 
procedure  primarily  by  means  of  the  risk  adjusted  discount  rate  approach, 
where  the  projected  outcome  is  modified  by  considering  new  financial  in- 
formation fe.g,,  how  well  is  the  project  justified,  who  support.;  the  project, 
etc.,)  and  by  introducing  a contingency  plan.  Concerning  iarnatment 
criteria,  Mao  found;  "Payback  period  is  primarily  a ctsk  measure. 

Accounting  profit,  since  this  is  what  the  financial  cooMunlcy  focuses 
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on,  is  especially  important  if  the  company  is  widely  held  and  rel.es  on 
external  sources  of  financing.  Internal  rate  of  return  is  most  likely  to 
be  the  major  criterion  in  closely  held  firms  which  are  less  worried  by  erratic 
patterns  in  their  per  share  earnings,  which  finance  themselves,  and  which 
make  many  small  investments  so  that  the  risk  in  any  one  investment  is  not 
critical."  [p.  359] 

Conrath  [3]  obtained  similar  results  via  a rather  simple  clinical 
experiment  wit  i line  business  executives.  Risk  was  perceived  to  be  related 
to  the  probability  of  failing  to  meet  a certain  target.  Additionally,  Conrath 
developed  a descriptive  model  of  decision  making  under  risk,  formulated 

mathematically  as  i 

C = f(L,p,e).  i 

Here  choice,  C,  is  depicted  as  a function  of  L,  the  internal  rate  of  1 

return  (or  more  generally,  its  utility)  below  which  one  is  viewed  as  having 
failed;  p,  the  probability  that  the  return  might  be  less  than  L,  and 
e,  the  expected  internal  rate  of  return. 

These  two  studies  suggest  that  the  overall  jrariakllity  of  the'  return 
distribution  (represented  in  most  theoretical  studies  jy  means  of  the  variance 
of  the  returns)  does  not  have  any  important  effect  on  capital  ^dgeting 
cisions.  They  rather  indicate  that  executives  focus  on  the  downside  part  " 
of  the  return  distribution.  In  order  to  shed  some  more  light  on  this  incon- 
gruence between  theoretical  hypotheses  and  empirical  finding^'  'e  conducted 
an  experi.aent , requiring  preference  judgments  from  business  executives  on 

hypothetical  Investment  alternatives.  >.  . • 


Other  research  questions  to  be  analyeed  in  this  study  Include;  (1)  How 
is  information  about  retuena  (represe'Ated  by-i  ROI  and  payback ''period)  aird 
abou.  other  aspects  of  the  investment  (as  number  of  additional  'ersonnel  to 
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be  hired  and  magnitude  of  the  investment)  combined  in  making  investment- 
decisions?  (2)  Do  upper  management  executives  make  different  investment 
decisions  than  middle  management  executives? 

However,  before  describing  the  experiment  and  its  results  we  will 
discuss  Several  methodological  issues  concerning  the  analysis  of  preference 


1 


judgments . 


2.  METHOD  FOR  ^ ’DGMENT  ANALYSIS 

The  most  commonly  used  methodology  for  examining  how  judges  combine 
several  information  cues,  on  a given  judgment  task,  into  an  overall  judgment 
is  a regression  analysis.  Such  an  analysis  has  been  applied  both  across 
multiple  decisions  of  a single  judge  or  across  decisions  of  multiple  judges. 
This  approach  requires  that  the  judgments  have  interval  scale  characteristics 
(l.e.,  order  and  difference  between  pairs  of  numbers  associated  with  the 
judgment  task  have  meaning).  F camples  of  several  judgment  tasks  studied  within 
the  traditional  regression  framework  are  as  follows: 

(1)  Predicting  academic  success  (grade  point  average)  of  graduate 
students  based  on  ten  variables  such  as  college  grade  point 
average,  peer  ratings  and  several  self  ratings  (Wiggins  and 
Kohen  [22]). 

(2)  Predicting  the  severity  of  Hodgkin’s  disease  on  a 9-point  scale 
based  on  biopsy  slides  (Einhorn  [5]). 

(3/  Determining  the  amount  willing  to  bid  for  playing  a specific 
bet  [Liechtenstein  and  Slovic  [11]). 

(4)  Rating  of  graduate  students  on  a five  point  scale  based  on 


information  available  for  three  variables:  the  graduate  Record 


-5- 


Exam,  undergraduate  grade  point  average,  and  an  approximate 
rating  of  the  quality  of  the  institution  at  whicn  tne  grade 
point  average  was  obtained,  (Dawes  [4]). 

(5)  Periodical  decisions  on  production  and  workforce  levels  by 
production  managers  for  an  experimental  task  (McCann,  Miller 
and  Moskovitz  [12]). 

All  of  these  Judgment  (decision  making)  tasks  require  basically  an  Interdlmen- 

sional  evaluation  strategy,  i.e.,  evaluating  each  alternative  separately  in 

terms  of  an  overall  criterion.  Studies  by  Russo  and  Rosen  [17]  and  Russo  and 

Dosher  [16],  however,  found  that  for  several  multi  alternative  .choice  tasks, 

* 

the  majority  of  subjects  did  not  use  such  a strategy.  Indeed,  they  found 
that  most  subjects  applied  an  intradimensional  evaluation  strategy,  i.e,  com- 
paring alternatives  on  several  choice  dimensions,  and  then  combining  the  re- 
sults of  these  comparisons  into  preference  judgments.  (A  similar  choice  strategy 
is  the  additive  difference  r.  del  suggested  by  Tversky  [20]),  In  general  their 
findings  suggest  that  subjects  (1)  partition  a multiple  choice  task  into  a 
sequence  of  paired  comparisons,  and  (2)  use  a intradimensional  evaluation  strategy 
within  these  comparisons  whenever  dimensional  comparisons  are  easy  to  perform. 

These  results  have  been  supported  by  Fidler  and  Atkin  [7],  who  found  that,  for 
a given  set  of  alternatives,  subjects  perceive  interdimensional  evaluation  judg- 
ments to  be  significantly  more  difficult  to  make  than  paired  comparison  judgments. 
In  addition,  other  studies  requiring  interdimensional  evaluation  strategies  (e.g., 
Tversky  aud  Kahneman  [19])  found  that  subjects'  conditional  probability  judgments 
were  generally  incorrect. 

Given  that  interdimensional  strategies  appear  to  be  difficult  to  apply,  less 
likely  to  be  used  than  intradimensional  strategies,  and  incorrect,  it  is  not 
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surprising  that  a linear  model  o£  the  judges'  responses  generally 
predicts  the  "true  judgment  criterion"  better  than  the  actual  judgmrnts 
Besides  the  fact  that  judgments  made  in  terms  of  interval  scale  responses 
are  likely  to  be  incorrect,  Che  variance  explained  by  linear  models  Is  often 
very  low.  (e.g.  Einhorn  [5]).  These  results  can  be  Interpreted  in  tiro  ways: 
either  Che  models  of  Che  judgment  process  i^re  not  very  good  or  the  judgments 
are  very  "noisy". 

In  order  "o  overcome  some  of  these  shortcomings  we  propose  to  estimate 
the  judges'  decision  rules  based  on  paired  comparison  preference  judgments. 
Since  pairwise  evaluation  strategies  are  simpler  and  more  common  we  would 
expect  to  get  better  models  of  the  judgment  behavior  than  the  studies  cited 
above. 

Model  of  Behavior 

Before  presenting  our  formal  choice  model  we  have  to  introduce  a few 
definitions.  First,  denote  s 

I = {1,2,... ,n] 

a set  of  n stimuli  for  which  dominance  judgments  have  to  be  made.  These  n 
stimuli  are  assumed  to  be  described  by 

J = {1.2,. ..,m] 

a set  of  m attributes.  Then  define  as 


the  i-th  stimulus  described  in  terms  of  m attribute  dimensions.  Further- 
more let  Q = {(k,!,);  k, 1,611  be  the  set  of  actual  paired  comparisons,  and 


define 


I 


¥(k,-t)  e n. 


Now,  our  model  can  be  formalized  as  follows.  Each  executive  S (where 
1 < S < N , and  N_  is  the  total  number  of  judges)  is  as.suraed  to  have  an 

O b 

utility  function 


^Skt  * 


which  is  dependent  on  the  attribute  differences  for  a given  binary 

choice  between  alternatives  k and  >t.  In  this  function  l^sk-t^’ 
absolute  value  of  the  utility  function  indicates  the  strength  of  preference 


for  either  alternative,  while  the  sign  of  reflects  whether  k > -t, 

i.e.  alternative  k is  preferred  to  alternative  t (when  ^ 

> k (when  < 0).  Furthermore,  we  assume  that  Ugj^^  consists  of  a 

deterministic  component  Vgj^^  which  is  a function  of  and  a stochastic 

error  t 


spe< 


E , , assumed  to  be  independent  of  V . and  capturing  all  mis- 
13  related  with  Therefore,  we  can  write 


^Sk^  '^Sk-t  ^Sk-C 


(1) 


where  V , is  assumed  to  be  approximated  by  the  linear  function 


'Sk^  = Aj^^  P . 


In  this  equation  B = represents  a set  of  choice  parameters  which 


determine  the  Judges'  preferences. 

Although,  our  data  collection  technique  results  in  Ugj^^  being 
directly  unobservable,  we  can  obtain  predictions  of  it  based  on  estimates 


of  these  choice  parameters. 
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In  addition,  we  assume  that  is  the  same  for  a given 

population  of  judges.  Then  we  can  rewrite  (1)  for  such  a population  as 


(!') 


This  assumption  seems  to  be  justified  because  several  independent  studies 
found  that  "given  enough  clinical  judges,  ill  possessing  at  least  some  validity, 
the  most  accurate  predictions  (in  situations  where  criterion  information  is 
lacking)  may  w.  T*  come  from  the  composite  judgments  of  the  total  group" 

(Goldberg  f9j,  p.  431)  (see  also  Wiggins  and  Kohen  [22]  for  similar  findings). 

Under  the  assumption  that  V . is  a linear  function  of  A , i.e. 

iC'v  K'C 

= Aj^^p,  we  can  rewrite  this  deterministic  part  of  our  choice  model  as 


• V*  ■ - V ''k  - 


In  this  equation  and  are  equivalent  to  the  deterministic  component 

in  linear  utility  functions  f the  form 


Uf  = Vf  + ef  Vi  e I 

where  e^  is  the  stochastic  component  of  the  utility  function  associated 
with  alternative  i.  This  means  that  if  is  a linear  function  of  A^^/ > 

the  inter-  and  intradlmensional  utility  models~are  observationally  eqi/ivaleof. 
Therefore,  we  ere  also  able  to  derive  predicted  utilities  for  the  choice  alter-- 
natlves  from  our  choice  model,  without  requiring  the  decision  ir.aker  to  use  a 
interdime. .sional  evaluation  strategy. 

Es^mation  Techniques 

tWo  T nhrs  irtr  *ii\g  live  pS  H.UBV'VVK*e>4  sKvvie  will  he 

described  now.  The  main  differences  between  these  two  techniques  lie  in  the 
algorithm  used  for  determining  the  attribute  weights  and  in  thi  assumptions 


about  the  error  terms. 
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A linear  programming  (LP)  method  proposed  by  Srinivasan  ard  Shocker  [18] 
is  purely  descriptive  and  makes  no  probabilistic  assumptions  about  the  error 
terms.  Its  goal  is  simply  to  estimate  the  9's  in  such  a way  that  the  pre- 
dictions of  the  model  are  as  consistent  as  possible  with  the  observed  judgments 
Formally  this  means  that,  for  a given  paired  comparison  judgmeot,  the  following 
relation  should  be  fulfilled: 


V^^>0 


for  all  (k.-t)  c n. 


where  V, , > 0 can  be  rewritten  either  as 
k-t  — 


(Xj^  - x^)  P > 0 or  0 for  all  (k.-t)  e 


Q. 


Now  let  us  define  for  all  Xj^  > X^ 


= < 


0 for  V,  . > 0 
k-C  — 


as  the  extent  to  which  Vj^  is  smaller  than  V^.  Then  for  any  particular  set 
of  weights  p we  can  determine 


(k,-L)eQ 


<V) 


which  is  called  the  poorness  of  fit.  Minimizing  this  poorness  of  fit  measure 
with  respect  to  p is  therefore  equivalent  to  the  goal  of  maki”'g  the 
as  consis^  .it  as  possible  with  the  judgments. 

Since  by  definition  nonnegative,  it  follows  that  the  trivial 

solution  0=  (0,0,..., 0)  would  always  minimize  the  poorness  of  fit  measure. 

In  order  to  preclude  this  solution  from  being  feasible  Srinivasan  and  Shocker 
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propose,  without  loss  of  generality,  to  impose  the  following  constraint; 


(k.-DeO 


By  defining  = (Vj^^)  we  can  write  the  linear  program  for  estimating  the 

p's  proposed  by  Srinivasan  and  Shocker  as  follows: 


min 


(k,i)»,Q 


k-t, 


such  that 


DP 


= 1 


^ 0 
< 


for  all  (k,'t)en 
where  D = S A 

(k,^)en 


As  already  mentioned  above,  this  technique  is  only  a de  '•/iptive  method 
for  estimating  the  p's,  without  any  probabilistic  assumptions  about  the 
distribution  of  the  error  terms.  Therefore  it  is  also  impossible  to  do  any 
statistical  analysis  of  the  estimates.  For  this  reason  Fidler  [6]  proposed 
a maximum  likelihood  (ML)  estimation  technique  for  analyzing  paired  comparison 
judgments.  The  assumptions  of  the  ML  method  can  be  summarized  as  follows; 


"skt  ■ + 


for  k=m,  -6  = 0,  S = t 
L 0 otherwise 


■Vk,'(,,m,n  e I and 


t S,t:  1 < S,t  < Nj 


(2) 


As  indicated  above  is  not  directly  observable  but  only  a categorical 


variable  Z indicating  which  stimulus  is  preferre.  in  a given  pairwise  judgment. 
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The  observed  categorical  dependent  variable  Z can  be  represented 
conveniently  as  a dummy  variable  which  assumes  the  value  1 if  Xj^ 
the  value  0 if  X^  >•  From  this  it  follows  that 


Z,  = 1 <=>  U,  , > 0 
k-c.  k-c. 


Based  on  equations  (2)  and  (3)  we  can  write  directly  the  probability 
function  of  the  observed  dependent  variable  Z as 

■ <’>  ■ f'' V < “>  ■ '■'<^(.6  ^ °>  = ''■■("kl  < 

■ ‘ 1 - * (-¥-)  <'*=> 

Pr<z^^  ■ » - '“'^(‘'kt  > <»  ■ ^ “>  ■ - V® 

- 1 

where  f(t)  represents  the  cumulative  standard  normal  density  function,  i.e., 


Ht)  ^ S . 

-00  ■/  2tt 


. X^/2 


and  is  the  probability  of  event  , 


— ^ Since  the  subscript  s for  the  different  judges  is  not  necessary  for  the 
further  discussion  of  this  technique  it  is  deleted  in  all  e''uations  below. 
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The  likelihood  of 
L - L(z|p^,. 


Z,  conditional  on  the  parameters  3,  is  then 

Z ( ' 

..B^)  = n [I$(A..3)]  ^[1-$(V.B)] 

(k,^,)ea 


ff 

and  the  log  likelihood  function,  L , is 


* 

L 


log  L = 


(k,-t-)eQ 


+ (I-Vti-’CVS)” 


(5) 


By  differentiating  (5)  with  respect  to  the  3's  and  equating  the  partial 
derivatives  to  zero  one  obtains  a nonlinear  equation  system  from  which  the 

A 

parameter  estimates  3 can  be  derived  numerically  by  an  iterative  procedure 
(e.g.  the  Newton  Raphson  method).  From  a theoretical  viewpoint  the  main 
advantage  of  the  ML  technique  lies  in  the  nice  statistical  properties  of  the 
ML  estimators.  Under  rather  general  conditions  the  estimates  are  consistent 
and  asymptotically  efficient,  and  their  asymtotic  sampling  distribution  is 
known.  In  addition,  hypotheses  can  be  tested  by  using  either  this  sampling 
distribution  or  by  means  of  the  likelihood  ratio  test.  On  the  other  hand,  how- 
ever, the  LP  method  does  not  require  any  specific  assumptions  about  the  error 
terms,  and  therefore  is  more  generally  applicable.  Therefore,  in  the  result 
section  we  will  compare  the  two  estimation  techniques  in  terms  of  their  ability 
to  predict  actual  judgments. 


3 . EXPERIMENTS 
Alternatives 

Twenty  hypothetical  investment  projects  were  developed.  Each  of  these 
alternatives  was  characterized  in  terms  of  four  attribute  dimensions;  Discrete 
probability  distribution  of  return  on  investment  (ROI),  payback  period  (PP), 
magnitude  of  the  investment  (MI),  and  number  of  additional  personnel  to  be  hired 
Each  alternative  was  characterized  by  one  value  on  each  of  thes  dimensions, 
except  for  ROI  where  three  possible  values  with  their  associated  p '^abilities 
were  specified. 
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In  order  to  analyze  how  executives  evaluate  probabilistic  infortantion, 
various  parameters  of  this  return  distribution  were  calculated  for  each  in- 
vestment project.  These  parameters  were  as  follows:  the  mean,  variance  and 
standard  deviation  of  the  ROl  distribution,  the  probability  of  not  meeting  at 
target  return,  and  the  below  target  ret  irn  semi  var lance. 

Subjects 

The  subjects  were  members  of  two  different  executive  development 
programs.  Experiments  1 and  2 consisted  of  n^^  ■ 29  and  1X2  ■ 33  subjects 
respectively.  Participation  was  voluntary.  For  both  experiments,  the  participants 
were  identified  as  either  upper  or  middle  management. 

Experimental  Design 

Each  subject  was  given  a questionnaire  containing  Instructions  and  a 
random  sample  of  thirty-four  pairs  of  investment  projects.  This  procedure 
resulted  In  approximately  5 subjects  per  experiment  judging  any  particular 
pair.  A preferable  method  w uld  have  presented  all  possible  pairs  (190)  to 
all  subjects,  but  this  was  deemed  infeasible  due  to  time  and  fatigue  factors. 
Results 

The  presentation  of  the  results  of  the  experiments  will  be  divided  into 
four  parts.  First  we  will  discuss  differences  in  the  results  between  the  two 
estimation  techniques  described  in  the  methodology  section.  Second  we  will 
examine  the  effects  of  management  position.  . In  the  third  section  we  will  ad- 
dress the  Issue  of  whether  executives  tend  to  focus  more  on  t^e  downside  or  on 
the  overall  variation  of  the  return  distribution.  Finally  we  will  discuss  dif- 
ferences between  the  two  experiments. 

^ The  evBierl^l  values  of  2S\ese  varlablea  dime  «ach  «ljeexBa£l(va<  ;mni-jdeacrll>cd 
in  TJmble  1.-  ?. 
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The  results  of  the  two  experiments  will  be  presented  In  terms  of  the 
estimated  attribute  weights  descriptive  criteria  of  fit,  and  the  rank- 

A,  A 

order  of  the  alternatives  Implied  by  the  estimated  utilities 

Both,  standardized  and  unstandardized  weights  are  displayed  In  the  tables 

because  they  have  different  Interpretations.  The  understandardized  weights 

A 

3^  indicate  the  estimated  change  in  utilicy  for  a unit  change  in  the  invest- 
ment  attribute  j,  while  the  standardized  weights  B^,  can  be  Interpreted 
as  the  relative  partial  saliences  of  the  respective  Investment  attributes. 
Mathematically  Is  defined  as: 

c ■■  Vj 


j m 

S I P,  a, 


j j 


where  Is  the  unstandardized  weight,  Oj  is  the  standard  deviation  and  6^ 

Is  the  standardized  weight  of  the  j-th  attribute  dimension.  The  number  of 
dimensions  Is  denoted  by  m and  { | refers  to  the  absolute  value  symbol. 

As  the  criterion  of  fit,  we  created  two  descriptive  Indices,  which  can 
be  used  to  compare  different  choice  models  and  different  estimation  techniques. 
These  two  Indices  are  the  percentage  of  correct  Individual  and  aggregate  pre- 
dictions. The  percentage  of  correct  individual  predictions  Is  calculated  as 

100  • correct  Individual  paired  comparison  predictions 

Total  number  of  Individual  paired  comparison  Judgments 

An  individual  prediction  of  a given  Judge  for  paired  comparison  (k,^)  is 

classified  as  correct  when  >■  and  >0  is  fulfilled.  On  the  other 

hand,  the  percentage  of  correct  aggregate  prediction  is  defined  as 

100  • correct  aggregate  paired  comparison  predictions 

^A  ” Total  number  of  aggregate  paired  comparison  Judgments 

An  aggregate  Judgment  in  this  context  focuses  on  all  individual  Judgments  made  for 


a given  paired  comparison  Then  an  aggregate  prediction  is  called 

to  X,  and  U,  , > 0.  Further- 
K,  k<{, 

more,  since  usually  not  all  Judges  make  Identical  Judgments,  the  upper  limit  for 
Pj  Is  leaeitivis  100.  Therefore  "we—are  aj^a .reporting  ^ {gAiM»eigl 

carrect.tndtvi^vel  >r«dictlone  Pp  which  i(g,.the  uglier  of 

re£leee»vthe‘hfinogeaelty  aiDMC^  the  Judges'  choices. 

Comparison  of  the  Estimates  Obtained  by  Both  the  ML  and  the  LP  Technique 

The  evaluation  of  differences  in  the  estimated  utility  functions  between 

the  two  estimation  techniques  will  be  presented  only  for  experiment  1 because 

the  results  from  experiment  2 are  highly  similar.  Of  the  29  participants  In 

experiment  1,  20  were  classified  as  middle  management  and  9 as  upper  management. 

Table  2 presents  the  Model  1 utility  functions  estimated  by  the  LP 

method  for  both  management  groups,  while  those  obtained  by  the  ML  method  are 

presented  in  Table  3.  Model  1 characterizes  the  probability  distribution  of  ROI 

by  its  mean  ^ and  its  stana^rd  deviation  a.  In  the  statistical  literature 
2 

a and  a are  used  Interchangeably  because  there  Is  a one  to  one  relationship 

between  the  two  measures.  From  an  behavioral  point  of  view,  however,  there  is 

2 

a difference  between  a and  a because  they  imply  a different  functional 

form  of  the  individual  utility  functions.  Therefore,  we  also  estimated  a variant 

2 

of  Model  1 by  using  a instead  of  a.  Although  the  latter  variant  resulted 
in  slightly  worse  predictions,  the  results  of  both  were  very  similar.  Therefore, 
we  have  onlv  reported  the  results  for  Model  1. 

A comparison  of  the  parameter  estimates  of  the  utility  function  and  the 
rankordering  of  the  alternatives  derived  from  the  estimated  utilities  reveals  that 
there  are  only  slight  differences  in  the  results  between  these  two  estimation 
techniques.  This  is  also  supported  by  the  correlation  coefficients  (1)  and  (2) 
in  Table  9 between  predicted  utilities  derived  from  the  parameti  estimates 


correct  if  the  majority  of  Judges  prefers  Xj^ 
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by  both  methods.  However,  from  a theoretical  point  of  view,  the  ML  method 
is  more  attractive  because  It  does  not  force  m-1  pairs  of  stimuli  to  have 
the  same  value  on  the  utility  scale  (see  Fldler  [6]  for  a more  thorough  dis- 
cussion of  this  deficiency)  and,  in  addition,  it  allows  statistical  analysis 
of  the  estimates.  Furthermore,  the  ML  method  consistently  predicted  more 
(although  only  slightly  more)  actual  judg.’.ents . This  predictive  superiority 
of  the  ML  method  probably  results  from  the  lack  of  "forced  ties"  between 
alternatives.  For  these  reasons,  we  will  only  consider  the  ML  method  in  the 
remainder  of  the  paper. 

Comparison  of  Upper  and  Middle  Management  Judgments 

Based  on  our  prior  beliefs,  we  expected  positive  attribute  weights  for 
ROI  mean,  and  negative  weights  for  PP,  ROI-a,  and  AP  both  for  upper  and 
middle  management  executives.  The  reasons  for  the  hypothesized  signs  of 
these  attribute  weights  are  quite  straightforward  and  therefore  are  not  further 
elaborated.  For  the  sign  of  MI  we  did  not  have  any  strong  prior  expectations, 
because  there  might  not  be  a monotone  relationship  between  MI  and  utility.  For 
example,  we  would  expect  for  low  ROI's,  MI  to  have  a negative  Influence  on 
utility  and  a positive  Influence  for  higher  ROI's. 

We  had  no  strong  prior  beliefs  related  to  whether  upper  and  middle  manage- 
ment executives  have  different  preferences  for  investment  alternatives.  Bower  [1] 
for  example  has  argued  that  managers  at  the  business  level  might  have  a more 
short  run  view  of  problems  because  they  move  every  two  to  three  years  but  on  the 
other  hand  might  also  have  more  of  a long  run  view  because  they  are  not  subjected 
to  public  and  stockholder  concern  with  quarterly  earnings  per  share. 

As  can  be  seen  in  Tables  3 and  4,  the  coefficient  estimates  have  the 
expected  signs  and  MI  has  a positive  sign.  In  order  to  test  for  a aonmonotone 


1 


I 


1 ii. 
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relationship  between  MI  and  utility,  we  divided  the  stimuli  into  two 
groups  based  on  the  mean  ROI.  Then  we  analyzed  each  of  these  subgroups 
separately,  but  also  found  a positive  relationship  between  MI  and  utility. 

Significance  tests  of  the  estimated  coefficients  show  that  in  all 
four  management  groups  the  null  hypothesis  Hq:  ~ 0 cannot  be  accepted 

at  the  significance  level  a “ 0.01  for  ROI-n,  PP,  and  MI.  For  ROI-a  and  AP, 
on  the  other  3^  ■ 0 cannot  be  rejected  sometimes  (a  ■ .05). 

A compa  standardized  weights  shows  that  ROI-p,  and  PP 

have  the  highest  pa^tia'.  effects  on  the  choices  between  pairs  of  alternatives. 
This  result  coincides  with  the  findings  of  the  survey  studies  cited  at  the 
beginning  of  this  paper.  Compared  to  these  two  variables,  the  other  three 
attributes  seem  to  have  a relatively  minor  effect  on  the  judgments. 

Since  the  estimated  attribute  weights  are  very  similar  for  upper  and 
middle  management  in  both  experiments  it  is  not  surprising  that  the  rankorder 
of  the  alternatives  based  on  the  predicted  utilities  is  not  very  different. 

This  is  also  confirmed  by  the  correlation  coefficients  of  = .98  and  r2  ■ .97 
between  the  predicted  utilities  of  upper  and  middle  management  executives  for 
experiments  1 and  2 (see  Table  9.1).  In  order  to  test  whether  the  choice  models 
for  upper  and  middle  management  derived  from  the  pair  comparison  judgments  were 
different,  we  performed  a likelihood  ratio  (Chi-Square)  test  for  each  experiment. 
The  results  of  these  two  Chi-Square  tests  are  given  in  equations  (1)  and  (2)  of 
Table  8.  F^'’-  both  experiments  the  test  statistics  indicate  that  the  upper 

management  has  the  same  utility  function  as  middle  management,  cannot  be  rejected 
at  the  confidence  level  a = .25.  Therefore  we  will  not  distinguish  between 
upper  and  middle  management  executives  in  the  following  analysis  which  examines 
whether  downside  variation  of  the  return  distribution  influences  judgments  more 


1 


than  overall  variations. 


mr 


-18- 


Downslde  vs.  Overall  Variations  of  the  Return  Distribution 

Several  authors  (e.g,  Mao  [13],  and  Conrath  [3])  have  recently  proposed 
that  business  executives  focus  on  the  downside  part  of  the  return  distribution 
rather  than  on  the  total  variation.  In  order  to  test  this  proposition,  we 
constructed  two  variables  reflecting  only  the  downside  part  of  the  return  dis- 
tribution. These  two  variables  were  the  below  target  return  semi-variance 
2 

(a^)  and  the  probability  of  not  achieving  the  target  return  (Pj.).  In  general, 
= E(x-T)^  for  X < T 

where  T is  the  target  return  and  E is  the  expectation  operator,  and 

= Pr(x  < T). 

2 

In  order  to  calculate  oip  and  P.^,  one  has  to  specify  the  target  return  T. 

Since  we  did  not  have  any  strong  prior  beliefs  about  the  size  of  the  target 

2 

return,  we  calculated  and  P^  for  target  returns  of  107.,  127.,  147,  and 

2 

167.  and  Included  each  different  and  P^  variable  (one  at  a time)  to  the 

Model  1 variables.  By  means  of  this  procedure  we  could  see  which  of  these  new 

variables  characterizes  the  downside  part  of  the  return  distribution  best.  The 

2 

results  of  this  procedure  suggested  that  inclusion  of  increased  the 

predictive  power  in  terms  of  and  P^.  Furthermore  T = 127.  was  the 

target  return  which  maximized  the  likelihood  function  for  the  different  T's 
and  had  the  highest  indices  of  fit.  Based  on  these  results  our  Model  2 pre- 

2 

sented  in  Table  6 contains,  in  addition  to  the  variables  in  Model  1,  also 
for  T = 12%. 

The  Model  2 results  for  both  experiments  show  that  the  partial  importance 
2 

weights  of  are  much  higher  than  those  of  ROI-a.  Furthermore,  tests  of  the 
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hypothesis  HqI  = 0 indicate  that  this  hypothesis  cannot  be  accepted 

2 

(a  = .005)  for  the  coefficient  estimates,  but  it  cannot  be  rejected 

(a  .25)  for  the  ROI-a  coefficient  estimates.  The  Indices  P and  P 

X A 

which  are  about  1%  higher  for  Model  2 than  for  Model  1 also  indicate  that 
Model  2 is  a somewhat  better  representation  of  the  Judgments.  These  results, 
obtained  from  a much  larger  sample  of  executives  compared  to  the  studies  by 
Conrath  [3]  and  Mao  [13],  strongly  support  and  advance  their  findings  of  how 
managers  evaluate  probabilistic  information  about  the  return  distribution. 

Thus  it  becomes  more  and  more  evident  that  the  overall  variation  of  the  return 
distribution  is  not  a good  construct  for  explaining  investment-decisions. 
Differences  in  Judgments  between  Experiment  1 and  2 

By  comparing  the  results  of  the  Judgment  analysis  between  experiment  1 
and  2 we  see  that  both  the  coefficient  estimates  and  the  rankorder  of  the  alter- 
natives are  almost  Identical.  This  similarity  Is  reflected  by  a Chi-Square  test 
of  Hq:  the  attribute  weights  are  equal  for  Che  two  experiments,  (see  Table  8, 
equation  (3)).  The  result  of  this  test  indicates  that  Hq  cannot  be  rejected 
at  the  significance  level  a = .3.  The  correlation  coefficient  r = .98  (see 
Table  9,  equation  (5))  between  the  predicted  utilities  of  experiment  1 and  2 
also  upports  the  results  of  the  Chi-Square  test. 

These  results  combined  with  the  results  obtained  from  the  separate  analysts 
of  the  upper  and  middle  management  Judgments  indicate  that  there  are  no  statis- 
tically significant  differences  between  the  linear  decision  rules  inferred  from 
the  paired  comparison  Judgments  of  various  groupings  of  Judges.  However,  this 
similarity  of  the  Judgment  Is  unlikely  due  to  effects  of  the  participation  In 
the  executive  development  program  because  both  experiments  have  been  conducted  at 
the  beginning  of  each  program,  before  any  lectures  on  capital  budgeting  were  given. 
It  rather  Indicates  that  business  executives  have  similar  preferenc  with  respect 


to  outcomes  of  Investment  decisions. 
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DISCUSSION 


The  results  of  this  study  confirm  and  extend  findings  from  other 
empirical  studies  of  capital  budgeting  decisions.  It  confirms  the  results  i 

of  several  other  studies  that  (1)  ROI  and  PP  are  the  most  Important  Invest- 
ment characteristics  and  (11)  that  dovmslde  variation  of  the  return  dlstrl- 
busines  has  a stronger  Influence  on  investment  decisions  than  overall  variation. 

The  approach  used  for  analyzing  capital  budgeting  decisions  in  this  study, 

however,  is  different  and  allows  a more  rigorous  analysis  of  the  decision  process.  i 

First,  we  postulated  a specific  choice  model  and  second  we  statistically  ^ 

I 

estimated  its  parameters  based  on  actual  decisions.  This  allows  us  to  determine  ^ 

the  partial  saliences  of  the  Investment  characteristics  and  the  combination  > 

rule  for  combining  the  information  cues  into  preference  Judgments.  Although 

we  expected  ROI  and  PP  to  be  the  most  Important  Investment  characteristics,  ’ 

! 

we  were  surprised  that  they  are  almost  equally  Important.  This  result  is  ^ 

compatible  with  explanations  that  payback  period  is  viewed  as  a measure  1 

of  (1)  the  risk  of  lost  opportunities  (Byrne,  Charnes,  Cooper,  and  Kortanek  [2])  | 

and  (2)  the  liquidity  and  uncertainty  of  an  asset  (Weingartner  [21]).  j 

I 

The  methodology  for  analyzing  organizational  decisions,  proposed  in  this  ' 

paper  is  different  from  most  other  studies.  The  judgments  required  from  subjects 
are  less  difficult  (yes-no  decisions  on  pairs  of  alternatives  instead  of  in- 
terval scale  responses)  and  therefore  most  likely  more  accurate.  The  underly- 
ing assumption  of  this  hypothesis  is  that  tradeoff  of  quality  (accuracy)  of  in- 
formation a'ainst  quantity  (how  much  different  are  the  alternatives)  enables  us  to 
obtain  a better  model  of  the  judgment  process  and  better  predictions  of  the  "true" 
criterion  variable.  However,  this  hypothesis  has  not  been  tested  yet,  but 
efforts  are  currently  underway  to  investigate  this  assumption. 


i n 


4 


Ik 


Based  on  the  models  presented  in  Tables  3 through  7 one  can  see  that  cne 
predictive  quality  of  the  model,  as  measured  by  P and  P , Increases  with 
the  number  of  judges.  With  increasing  number  of  Judges,  P^  comes  closet  to  f-^ 
while  P^  approaches  100.  Furthermore,  our  results  show  that,  although  the 
criteria  of  fit  increase  with  the  number  of  judges  the  model,  (in  terms  of  the 
linear  decision  rule)  on  the  other  hand,  does  not  change  significantly  with  m •■f 
judges.  This  seems  to  support,  as  a rule  of  thumb,  that  approximately  two 
judgments  for  each  paired  comparison  (which  corresponds  roughly  to  a group  o'  i<' 
judges  in  this  experiment)  should  be  sufficient  to  estimate  the  inferred  linear 
decision  rule  of  a homogenous  group  of  judges. 

In  general,  our  methodology  can  also  be  used  for  analyzing  individual 
judgments  and  to  infer  the  decision  strategy  used  by  one  decision  maker.  The 
inferred  decision  strategies,  then  could  be  used  for  determining  individual 
differences  among  individuals  and  might  help  to  develop  training  programs  to  min- 
imize individual  differences  in  decision  making.  However,  the  most  important 
potential  application  of  this  technique  is  as  a decision  making  aid.  Once  the 
attribute  weights  have  been  estimated  they  can  be  used  to  rankorder  not  only  the 
alternatives  for  which  the  decisions  have  been  made,  but  also  others  in  similar 
decision  situations.  Depending  on  the  importance  of  the  decision,  the  prediett^d 
utilities  can  be  used  either  as  the  decision  criterion  or  as  a screening 
technique  in  order  to  separate  the  "good"  alternatives  from  the  rest.  Then 
management  would  have  to  evaluate  only  a reduced  set  of  alternatives  and  could 
therefore  investigate  each  "good"  alternative  more  thoroughly.  The  question  re- 
mains whose  judgments  (one  expert  or  a group  of  experts)  should  be  used  for 
determining  the  attribute  weights.  For  this  issue,  of  course,  there  exists  no 


universal  answer  (unless  one  knows  the  best  expert  in  the  field  v' tn  certairit)), 
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but  there  exists  some  evidence  that,  in  general,  the  model  derived  from 
judgments  of  a group  of  experts  is  a better  predictor  of  the  true  criterion 
than  a model  derived  from  the  decisions  of  one  expert  (cf.  Goldberg  [9]). 
Therefore,  it  might  be  a reasonable  strategy  to  use  the  judgments  of  a 
committee  as  a basis  from  which  the  decision  rule  is  inferred. 

The  methodology  proposed  in  this  paper  for  analyzing  preferences  is 
based  on  evidence  generated  by  studies  of  individual  choice  behavior,  hmplri  l 
studies  will  have  to  show  whether  linear  decision  rules  derived  from  paired 
comparison  judgments  are  superior  to  the  ones  obtained  from  rankorder  or 
interval  scale  responses.  Additionally  it  will  have  to  be  tested  whether 
some  of  the  findings  such  as  the  bootstrapping  phenomenon  or  the  superiority 
of  the  composite  judge,  obtained  in  the  traditional  judgment  studies  are  also 
valid  if  the  type  of  judgment  to  be  made  is  less  complex.  This  type  of  .analys’s 
would  show  how  studies  of  the  cognitive  aspects  of  the  decision  making  process 
could  be  applied  to  Improve  organizational  decisions  and  to  reduce  the  cognitive 
strain  of  executives  by  freeing  them  from  making  time  cosactming' voutine  decXsica 
for  which  no  quantitative  criteria  are  readily  available. 
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Amount 

ROI 

ROI 

Project 

Payback 

of 

Add! tlonal 

ROI 

Standard 

Semi- 

Number 

Period 

Investment 

Personnel 

Mean 

Deviation 

variance 

1 

4.00 

100.00 

3.0C 

10.40 

1.95 

5.6 

2 

2.00 

40.00 

2.00 

16.80 

4.07 

0.8 

3 

5.00 

75.00 

6.00 

9.20 

2.57 

‘3.6 

4 

7.00 

150.00 

2.00 

13.20 

3,26 

] 1 

5 

6.03 

70.00 

8.00 

17.90 

2 02 

0. 

6 

1.00 

20.00 

0.00 

19.60 

2.32 

0. 

7 

6.00 

100,00 

9.00 

16.60 

4.20 

0.3 

8 

4.00 

80.00 

4.00 

11.60 

2.37 

3. 1 

9 

3.00 

60.00 

7.00 

11.40 

3.10 

3.2 

10 

8.00 

50.00 

1.00 

10.20 

2.24 

7.4 

11 

4.00 

60.00 

5.00 

14.30 

2.10 

0 1 

12 

1.00 

90.00 

6.00 

11.60 

1,55 

2.1 

13 

8.00 

150.00 

3.00 

16.10 

1.92 

0.  1 

14 

4.00 

70,00 

8.00 

17.60 

2.86 

0 

15 

7.00 

200.00 

2.00 

12.00 

2.05 

2.7 

16 

8.00 

100.00 

5.00 

12.20 

1.41 

0.8 

17 

3.00 

20.00 

2.00 

10.40 

2.49 

8.0 

18 

5.00 

75.00 

7.00 

13.60 

1.95 

0.8 

19 

3.00 

250,00 

9.00 

13.20 

2.57 

0,4 

20 

9.00 

150.00 

5.00 

12,90 

2.07 

1.2 

r 

Mean 

4.9 

95.5 

4.7 

13,5 

2.45 

2.6 

standard 

Deviation  2.4  58.6 
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Standardized  and  Unstandardized  Attribute  Weights 
Estimated  by  the  LP  Method  for  Experiment  1 


Upper 

Middle 

Upper 

Ml. 'die 

Management 

Management 

Management 

Manag’.mrn 

Standardized 

Standardized 

Unstandardized 

Unstand.irdl  zcl 

Weights 

Weights 

Weights 

Weights 

Payback  Period 

-.30 

-.32 

-.12 

Magnitude  of  Investment 

.08 

.10 

.0014 

00  1.7 

Additional  Personnel 

-.13 

-.16 

-.048 

- . ' ,9 

ROI-Mean 

.41 

.38 

.14 

.13 

ROI-Standard  Dev. 

-.08 

-.04 

-.11 

-.05 

Percentage  of  Correct 

Aggregate  Predictions 

75.1 

71.6 

Percentage  of  Correct 

Individual  Predictions 

79.6 

72.7 

Percentage  of  Potential 

Correct  Individual 
Predictions 

86.2 

81.0 

Rankorder  of  the  Investment  Projects  Implied  by 

these  Weights 

Upper 

Middle 

Management 

Managpinent 

Rank 

Project  Number 

Pro 

jec  t Number 

1 

6 

6 

2 

2 

2 

3 

14 

14 

4 

5 

12 

5 

6 

/ 

12 

iK 

© 

( 

13  ) 

11 

8 

\ 

13 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
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Standardized  and  Unstandardized  Attribute  Weights 
Estimated  by  the  ML  Method  for  Experiment  1 
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Table  7 

Standardized  and  Unstandardized  Attribute  Weights 
for  Pooled  Experiments  1 and  2 


Standardized 

Weights 


Unstandardized 

Weights 


Sccriderd 

Error 


Payback  Period 

-.30 

-.16 

.01 

Magnitude  of  Investment 

.14 

.0031 

,00046 

Additional  Personnel 

-.09 

-.044 

. ()()« 

ROI-Mean 

.30 

.14 

014 

ROI -Stand.  Dev. 

-.02 

-.045 

,05 

ROI - Semi - Var 1 anc  e 

-.15 

-.056 

Percentage  of  Correct 

Aggregate  Predictions 

92.6 

Percentage  of  Correct 

Individual  Predictions 

76.8 

Percentage  of  Potential 

Correct  Individual 

Predictions 

77.9 

Rankorder  of  the  Investment  Projects  Implied  by 

these  Weights 

Rank 

Project  Number 

1 

6 

2 

2 

3 

19 

4 

14 

5 

12 

6 

5 

7 

11 

- rv 

<1- 

8 

13 

w 

9 

7 

V 

j ^ 

10 

18 

y 

0 

11 

4 

» v* 

12 

15 

13 

8 

14 

9 

15 

1 

V 

16 

20 

17 

17 

18 

16 

19 

10 

20 

3 

I 

1 

J 
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Table  8 

Likelihood  Ratio  (Chi-Square)  Tests 
A.  Tests  for  differences  between  upper  and  middle  management 

V ” 5 

V = 5 


V ^1  “ ^1 

2)  Hq;  0^2  “ ^2 


B. 


Test  for  differences  between  experiments  1 and  2 
3)  Hg:  6^  =02  ‘ 


V - 6 


2 

X 


®ui 

®U2 

^1 

^2 

01 

02 


= value  of  the  Chi-Square  test 

2 

= degrees  of  freedom  of  x distribution 

■ upper  management  weights  for  experiment  1 
« Upper  management  weights  for  experiment  2 
= middle  management  weights  for  experiment  1 

■ middle  management  weights  for  experiment  2 

■ weights  of  all  executives  for  experiment  1 
“ weights  of  all  executives  for  experiment  2 


A 


j 

I 

I 
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Table  9 


Correlation  Coefficients  Between  Predicted  Utilities  U 


(1)  r 


0.979  where 


U^  - upper  management  ML  method  estimates 
ML 

Uj^p  ■ upper  management  LP  method  estimates 


middle  management  ML  method  estimates 

(2)  ’^0  u “ 0.970  where 

Mj^  M^p  M^p  = middle  management  LP  method  estimates 


(3)  r 


w 


. 984  where 


Uj  = upper  management  experiment  1 estimates 
Mj^  = middle  management  experiment  1 estimates 


(5)  rf 


,974  where 


.986  where 


U2  = upper  management  experiment  2 estimates 
M^  ~ middle  management  experiment  2 estimates 


1 > experiment  1 estimates 

2 = experiment  2 estimates 


d 


i t 

i 

f 

[ 2] 

[ 3] 

[ 4] 

[ 

( 

[ 5] 

i 

i 

t [ 6] 

[ 7] 
[ 8] 
[ 9] 

[10] 

[11] 

[12] 

[13] 
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Executive  decisions,  expersmental  decision  making,  capital  budgeting 
decisions,  paired  comparisons,  multiple  attributes,  composite  criterion 
model. 


20.  ABSTRA^  rCwillnua  on  )•■)•)••  tid*  II  nmadddtrt  Idmntllr  br  block  mt*<E«r> 

This  paper  analyzes  capital  budgeting  decisions  of  business  executive^ 
in  an  experimental  setting.  Two  different  data  analysis  techniques,  a linear 
programming  and  a maximum  likelihood  method,  are  used  to  analyze  preference 
judgments  for  hypothetical  Investment  projects.  For  each  of  the  twenty  in- 
vestment projects  the  following  information  was  provided:  a probability  distr^ 
bution  of  returns  on  Investment,  and  magnitude  of  the  investment,  the  number 

of  additional  men  to  be  hired  for  implementing  the  investment,  and  the  
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expected  payback  period  for  the  Investment.  Five  conclusions  are  demon- 
strated based  on  two  Independent  experiments  with  business  executives: 

O)  Mean  return  on  Investment,  and  payback  period,  are  the  most  Important 

''  Investment  characteristics; 

(2)  Uppei^  and*'‘ib.ddl(i  managSBMnt  executives  have  almost  Identical  preference 
BtruSturelj; 

(3)  Downside  yprletlen  of  the  return  distribution  has  a stronger  influence 
on  InvestdUint  decisions  then  overall^varlatlons; 

(4)  Usefulness  of  this  methodology  for  (^)  analyzing  preference  Judgments 

of  individual  executives;  (j^)  training  purposes;  and  QHl)  as  «n  aid 
to  organisational  decision  making;  3 

(5)  Both  data  analyals  techniques  give  highly  similar  conclusions. 
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